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District Capacity Assessment (DCA) Brief 


The purpose of this brief is to provide an overview of the District Capacity 
Assessment and provide resources for additional information and learning. 


Introduction 


School districts (local education agencies) in the United States are charged with many 
responsibilities, by law and by custom. The most important is responsibility for assuring schools 
and teachers have the support they need to provide consistent, high quality, and effective 
education to promote student learning for all students every year. Measures of student 
outcomes are now commonplace in education. Those measures, while important, have not led 
to improved student outcomes (Manna, 2008; National Center for Education Statistics, 2013). 
Recognizing the key role of teachers, what is missing is a measure of district supports for 
schools and teachers. The District Capacity Assessment was developed to fill this gap. 


The District Capacity Assessment (DCA; Ward et 
al., 2015) is grounded in implementation science es 


and the Active Implementation Frameworks (Metz 


& Bartley, 2012). Implementation science is the A DISTRICT IMPLEMENTATION TEAM 
study of factors that influence the full and (DIT) IS COMPRISED OF KEY LEADERS 
effective use of innovations in practice. The (E.G. DISTRICT SUPERINTENDENT), 
leadership, competency development, and LEAD TEACHERS, COMMUNITY 


MEMBERS AND LEAD DISTRICT AND 
BUILDING ADMINISTRATORS. 


organizational support factors identified in 
implementation science have been built into the 
26 items that comprise the DCA. These factors are 
used by district staff to support schools and teachers’ use of evidence-based interventions and 
strategies that lead to positive student outcomes. 


The DCA is completed by district leadership and members of the District Implementation Team 
(DIT). The District Implementation Team uses the results of the DCA (scores, notes from 
discussion) to guide action planning. Action planning is focused on the development of 
infrastructure and capacity to support effective education practices across schools. Although 
the primary purpose of the DCA is to assist school districts in using evidence-based 
interventions and strategies, the DCA also fills a void by providing an avenue for collecting data 
to guide and monitor the development of infrastructure to support the use of evidence-based 
interventions and strategies. 


Items on the DCA assess the development of capacity in terms of the Active Implementation 
Frameworks (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005). The “capacity” of a district to 
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facilitate building-level use of evidence-based interventions and strategies refers to systems, 
activities, and resources that are necessary for schools to adopt and sustain effective 
innovations (Ward et al., 2015). Key systems, activities, and resources required for strong 
implementation and sustainability of efforts are identified across items that are comprised 
within three Implementation Drivers Subscale Scores: Leadership, Competency, and 
Organization (see Table 1). 


The DCA is an action assessment. That is, data from the DCA are used to guide action planning 
and next steps in an effort to close discrepancies between the capacity needed to adopt and 
sustain use of evidence-based interventions and strategies with fidelity and the current capacity 
detected by the DCA. As a result, the DCA is not considered to be completed until an 
implementation capacity action plan has been developed and reviewed. 


Table 1. District Capacity Assessment (DCA) and Subscales 


Leadership - Active involvement in facilitating and sustaining systems change to support 
implementation of the effective innovation through strategic communication, decisions, 
guidance, and resource allocation. 


Leadership — District Implementation Team (DIT) exists with leadership and funds to 
support implementation 


Planning — Plans exist and are monitored to guide implementation 


Competency - Strategies to develop, improve, and sustain educators’ abilities to use 
evidence-based interventions and strategies as intended in order to achieve desired 
outcomes. 


Performance Assessment — A fidelity measure is supported to guide on-going staff feedback 
Selection — A process for selecting staff is used 
Training — DIT secures and evaluates training 


Coaching — A coaching service delivery plan is used and coaching effectiveness is evaluated 


Organization — Strategies for analyzing, communicating, and responding to data in ways that 
result in continuous improvement of systems and supports for educators to use evidence- 
based interventions and strategies with good outcomes 


Decision Support Data System — DIT has access to data for the innovation that supports decision 
making at the district and building levels 


Facilitative Administration — DIT has formal processes for meeting, communicating, linking 
innovations to efforts, and addressing internal barriers 


Systems Intervention — DIT uses a process to report policy relevant information to outside entities 
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A Cascading System of Implementation Supports 


As shown in the graphic below, a cascading system of | 
supports consists of linked Implementation Teams that 


align, integrate, and leverage existing resources in a THE ESSENTIAL PURPOSE OF A 
state education system. Implementation teams at all DISTRICT IMPLEMENTATION TEAM 
levels are charged with developing the infrastructure to — |S TO DEVELOP, IMPROVE, AND 
support work at school and classroom levels so that SUSTAIN EFFECTIVE SUPPORTS AT 
high quality education practices can be delivered as Tee BEDI REVEL Unie 


BUILDING IMPLEMENTATION TEAMS 
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DELIVERING INNOVATIONS OR 
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intended. If barriers arise that are beyond a district’s 
capacity to address, the work of the District 
Implementation Team is to lift those issues up to 
regional and state levels. Systemic resolutions to 


Observation Tool for Instructional Supports and Systems (OTISS) 


Teacher Instruction and Student Outcomes 


Drivers Best Practices Assessment (DBPA) 


Building Implementation Team 


District Capacity Assessment (DCA) 


District Implementation Team 


Regional Capacity Assessment (RCA) 


Regional Implementation Team 


State Capacity Assessment (SCA) 


State Implementation Team 


district and school issues result in improved supports for existing teachers. For example, a 
review of DCA data might highlight gaps in Facilitative Administration and Systems Intervention. 
Given that these items tap into a district’s power to address barriers that impede effective use 
of an innovation, planning that identifies actions at both the district and regional levels should 
follow. In this scenario, district planning may address barriers in the flow of useful information 
by aggregating teachers’ concerns at the building and district levels. Simultaneous action 
planning at the regional level may revisit the flow of information between the district and 
regional team. As such, DCA data serve to inform current status, facilitate action planning, and 
monitor systems, actions, and resources that support implementation capacity over time. 


Page 3 


District Capacity Assessment (DCA) Brief 


Technical Information 


Development of the DCA was informed by an extensive review of the literature. Its 
psychometric properties have been examined with respect to usability, feasibility, construct 
validity, and reliability (Russell et al., 2016). The DCA Total Score has strong internal consistency 
with a Cronbach's alpha coefficient of .91. The three subscales also have adequate internal 
consistency: Leadership (a= .79), Staff Competency (a= .79), and Organizational Systems (a= 
.81). The assessment has been found to be sensitive to growth or decline in implementation 
capacity (Russell et al., 2016). Additional efforts are underway to assess the consequential 
validity of the DCA: that is, how the instrument is used in practice and the relationship between 
DCA scores and indicators of school and teacher performance. 


Administration 


The DCA takes one to two hours to complete. It is led by a trained Administrator who meets 
with a team comprised of district leadership and other staff intentionally selected for their roles 
in supporting use of the innovation (e.g., District Implementation Team). The Administrator 
guides the group through discussion and scoring of items and later action planning. Additional 
roles include a Facilitator and a Note Taker. The Facilitator is very familiar with the district and 
contextualizes items during scoring and discussion. The Note Taker records scores and 
discussion that arises during the scoring process. Using a standardized voting process, items 
are scored along a three-point scale as fully in place (a score of 2), partially in place (a score of 
1), or not in place (a score of 0). A scoring rubric provides key indicators for assigning scores to 
items. 


DCA Use as Intended 


The DCA is administered twice a year, in October/November (set action agendas for the year) 
and again in February/March (a strategic time for district budgeting and planning). Two 
administrations per year help district leaders and others monitor progress and engage in 
continued action planning to improve district and regional infrastructures for supporting 
implementation of effective innovations. 


Figure 1 depicts DCA data collected over time in one district. The DCA scores from the October 
administration indicate baseline district capacity in the 20% range. With the support of a 
Regional Implementation Team, this district engaged in purposeful action planning and 
attention to development of systems, activities, and resources related to Leadership, 
Competency, and Organization supports for schools and teachers in the district. By the end of 
the academic year, DCA scores had improved to the 60% range. The district used the second 
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set of scores to begin action planning for the fall with a third administration of the DCA 
scheduled to monitor progress. 


Figure 1. DCA Over Time 
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Figure 2 provides a broader picture of data collected across five districts served by one regional 
education agency (REA). These data provide a snapshot of the impact of REA support on the 
development of infrastructure and capacity to use innovations in the region. The baseline Total 
Score for each district is in the 20-40% range that is typical across hundreds of districts. Low 
Competency development scores (a score of 0 for 4 of the 5 districts) also are common across 
districts. The pattern of baseline scores is different for each district and the Regional 
Implementation Team uses these data to support the development of a District Implementation 
Team in each district. With this support it is likely that DCA scores will be in the 80-100% range 
within two years. The DCA informs action plans at the district level and at the regional level so 
each team can identify needs and allocate resources and supports in service of building 
sustainable use of evidence-based practices in their schools. For example, a local district used 
DCA scores in conjunction with fidelity scores to advocate for the school board to allocate 
funding for coaching of their identified innovation. With funding secured, the district was able 
to action plan and create a system to support effective coaching. 
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Figure 2. DCA Data Across Five Districts 
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Summary 


Data from the DCA serve a critical role in informing and developing support within a Cascading 
System of Implementation Supports. The DCA administration process allows both district and 
regional implementation teams to reflect on their current processes and practices in supporting 
instructional staff in their use of evidence-based interventions and strategies. Importantly, 
these results assist district and regional implementation teams in creating implementation 
capacity action plans that target needed infrastructure and supports to implement innovations 
with fidelity and produce intended outcomes. This task would be difficult, and perhaps 
impossible, without capacity assessment tools such as the DCA. 
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Learn More 


e The District Capacity Assessment 

e NIRN Implementation Measures 

e The Active Implementation Hub 

e The National Implementation Research Network 


This document is based on the work of the National Implementation Research Network (NIRN). 


© 2017 Dean Fixsen, Caryn Ward and Allison Metz 


This content is licensed under Creative Commons license CC BY-NC-ND, Attribution-NonCommercial-NoDerivs . You are free to 
share, copy, distribute and transmit the work under the following conditions: Attribution — You must attribute the work in the 
manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work); 
Noncommercial — You may not use this work for commercial purposes; No Derivative Works — You may not alter, transform, 
or build upon this work. Any of the above conditions can be waived if you get permission from the copyright holder, 
nirn@unc.edu. 


Page 7 


